Systems and methods for improved server-side contextual page analysis

ABSTRACT

Systems and methods are described for server-side contextual analysis of content available at a given uniform resource identifier (URI), which utilizes headless browser techniques to analyze a more complete and accurate version of page content than using existing techniques. For example, systems and methods are described for performing contextual analysis of content that would typically be displayed to a client device but is not included in an HTML file or other initial page source file available at the initially provided URI. The contextual analysis performed may include analyzing text using natural language processing and analyzing images using computer vision techniques.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No.62/946,302, filed on Dec. 10, 2019, the entirety of which is herebyincorporated herein by reference.

BACKGROUND

Publishers of various types of news, editorial or other content oftenchoose to present advertisements on their webpages, user interfaces, orother electronic content displayed to viewers of such content. Forexample, a news article presented in the form of a webpage presented toa user may include photos or other images within the webpage, text ofthe news article itself, as well as an advertisement included somewherewithin the page. Advertisements may be presented in any of a number ofknown forms or formats, such as embedded within (or overlaid on top of)a portion of an underlying image that appears on the page, as a banneradvertisement in its own portion of the page, as an interstitialadvertisement that temporarily covers at least a portion of other pagecontent, or as a pop-up advertisement in a separate window.Advertisements may be placed dynamically at the time of a page load by auser device as a result of the user's browser requesting anadvertisement from an advertisement service's server that is referencedin code previously placed within the page by a publisher of the page,for example. If a user viewing the webpage or other user interface makesa selection (such as by a mouse click or tap gesture) within theadvertisement, the user's browser or other application displaying thewebpage or other user interface may load a page associated with theadvertisement. This may occur, for example, by the browser sending anetwork request for content having a uniform resource identifier (“URI”)that was previously associated with the selected advertisement.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages will becomemore readily appreciated as the same become better understood byreference to the following detailed description, when taken inconjunction with the accompanying drawings, wherein:

FIG. 1A is an illustrative operating environment for performingcontextual analysis of content and selecting an advertisement fordisplay in association with the content, according to some embodiments.

FIG. 1B provides an illustrative data flow within the environment ofFIG. 1A, according to some embodiments.

FIG. 1C provides an illustrative data flow for performing contextualpage analysis using a contextual intelligence system as describedherein, according to some embodiments.

FIG. 2 is a flow diagram of an illustrative method for determiningwhether to perform contextual page analysis for a specific page prior toselecting an advertisement, according to some embodiments.

FIG. 3 is a system block diagram of a computing environment suitable foruse in various embodiments of the present disclosure.

FIGS. 4A and 4B illustrate a displayed page both prior to selection ofan advertisement and then subsequent to an overlay advertisement beingdisplayed over an underlying page image.

DETAILED DESCRIPTION

Generally described, aspects of the present disclosure relate toimproved systems and methods for server-side contextual analysis ofcontent available at a given uniform resource identifier (URI), whichutilizes headless browser techniques to analyze a more complete andaccurate version of page content than using existing techniques. Forexample, aspects of the present disclosure include contextual analysisof content that would typically be displayed to a client device but isnot included in an HTML file or other initial page source file availableat the initially provided URI. In some embodiments, the results ofcontextual page analysis described herein may be used to select anadvertisement or other supplemental overlay content that is dynamicallydisplayed in association with a page or other underlying content. Insome embodiments, the supplemental content may be displayed over orwithin an image, video or other page element that appears on a webpageor other user interface. In other embodiments, the supplemental contentmay be displayed as a banner advertisement, a pop-up advertisement, aninterstitial advertisement, and/or in some other form.

According to one embodiment, a computer-implemented method describedherein may include receiving, at a server, an application programminginterface (API) request to perform contextual analysis of a page,wherein the page is identified by a uniform resource identifier (URI),then initiating execution of a headless content extractor by the server,where the headless content extractor is configured to extract contentincluded in or referenced in the page. The extracted content may includetext formatted in a HyperText Markup Language (HTML) format and one ormore images and/or videos (depending on the embodiment), where at leasta portion of the content extracted was not included in an initial filestored at the URI but is instead requested by the headless contentextractor from an external system over a network based on at least oneof execution by the headless content extractor of code included in theinitial file or a reference in HTML of the initial file to an additionalURI or other network resource. The method may then include performingcontextual analysis of the content extracted by the headless contentextractor, where performing the contextual analysis may includeanalyzing text of the content using natural language processing andanalyzing any images and/or videos using one or more computer visiontechniques. The method may then include generating a response to the APIrequest that includes results of the contextual analysis, where theresults are based at least in part on the analysis of the text and theanalysis of the images and/or videos (if any appeared on the page).

A headless content extractor, as described herein, may be implemented atleast in part using headless browser functionality. As will beunderstood by one of ordinary skill in the art, a headless browser mayinterpret and process HTML or other code of a page in a manner similarto a standard web browser, but its output may be used for somethingother than display on a screen (e.g., interactions with the page may betested in an automated manner without any actual user interface beingpresented). For example, a headless content extractor as describedherein may receive an initial HTML page and then generate final HTMLcode based on a final state of a headless browser after loadingadditional content referenced within the page and executing associatedcode (such as JavaScript) included in the initial page content. In someembodiments, the headless browser operated in association with theheadless content extractor may include existing “ad blocker” features sothat content such as advertisements that may change frequently dependingon the user viewing the page is not loaded for purposes of thecontextual analysis that will be described herein (which is typicallynot user-specific, and may be reused across different users).

While reference will be made below to a “page” and a “browser,” it willbe appreciated that the disclosure herein is also applicable to contextsother than webpages and web browsers. For example, various otherapplications, such as applications executable on a mobile device, maypresent various user interfaces that include native images or videocontent on which overlay content as described herein may be presented.Various applications are configured to process HTML, including emailclients and social media applications, such that only minormodifications, if any, enable the above methods to be implemented in avariety of contexts other than a browser.

FIG. 1A is an illustrative operating environment 100 for performingcontextual analysis of content and selecting an advertisement fordisplay in association with the content, according to some embodiments.In the illustrated embodiment, an advertisement computing system 120 maybe in communication with a number of different user computing devices102 and publisher servers 103 over a network 108, such as the Internet.The advertisement computing system 120 may be operated in associationwith an advertisement service that different website publishers havepartnered with to present advertisements on pages of those publishers'websites.

The user computing devices 102 may be devices operated by users whorequest a URI from one of the publisher servers 103 for display via abrowser or other application operated on the user's device, which inturn may cause the browser to request advertisement content from theadvertisement computing system 120. The advertisement computing systemincludes an advertisement selection system 122, which may be local orremote communication with a data store 124 and a contextual intelligencesystem 130. The contextual intelligence system 130 may generally providethe advertisement selection system 122 with results of contextualanalysis of a given page that may be used in the advertisement selectionprocess, as will be further described herein. The contextualintelligence system 130 may include a contextual analysis API 136,headless content extractor 132, natural language processing component134, computer vision system 138, and data store 145, each of which willbe described further below with respect to other figures.

FIG. 1B provides an illustrative data flow within the environment ofFIG. 1A, according to some embodiments. The data flow begins with (1)the user computing device 102A requesting and receiving page contentfrom a publisher server 103A. For example, a browser or otherapplication operating on the user computing device 102A may request pagecontent associated with a URI hosted by the publisher server 103A, whichmay be returned by the publisher server 103A as an HTML file. Thebrowser or other application of the user computing device 102A may then(2) process the HTML and other code of the page (such as JavaScript orother scripts) to generate page content for display to the useroperating the user computing device 102A. In the illustrated example,the page may include a reference to the advertisement selection system122. For example, a JavaScript tag may have been included in the pagecode by the publisher of the page at the direction of an operator of theadvertisement service. In some embodiments, the tag may be placeddirectly in a webpage's HTML, such as in the footer of the HTML. Inother embodiments, the JavaScript may be loaded through a publisher's adserver. When the user device's browser loads the page, the JavaScript orother code placed within the HTML or other page code may cause thebrowser to request additional code from the advertisement selectionsystem 122, which may be considered a request for advertisement contentto be selected by the advertisement selection system 122 and returned tothe user device for integrated display within the page.

In response to the ad request being received as step (3), theadvertisement selection system 122 may (4) send a request for contextualinformation of the page to the contextual intelligence system 130, aswill be further described below with respect to FIG. 1C. The request tothe contextual analysis API 136 of the contextual intelligence systemmay include an indication of the URI of the underlying page to beanalyzed. The remainder of steps referenced in FIG. 1B may be in aninstance in which the contextual analysis system returns a result of acontextual analysis of the page to the advertisement selection systemsubstantially in real time with display of the page on the usercomputing device 102A. In some other instances according to someembodiments, the contextual intelligence system may add the page to aqueue to be analyzed such that no advertisement is served for the givenpage request from device 102A (such as if this is the first time an adhas been requested for the given URI), but where a later request for anad to be displayed on a displayed page associated with this same URI maybe returned quickly by the API 136 and an ad served in that secondinstance.

Once the API 136 returns responsive contextual information regarding thepage, the advertisement selection system 122 may (5) select anadvertisement to be displayed on the page based at least in part on thecontextual information. The contextual information may includeinformation such as various keywords associated with the page, eventcategories, content categories, threat categories or other brand safetyinformation, and/or other data.

Once the system 122 selects an advertisement or other overlay content tobe presented by the user device 102A, the system 122 may retrieve orassemble a package that includes one or more media assets (such asimages, video data and/or other visual media assets), along withadditional code that can be executed at the user device and which causesthe user device (in combination with code that may have been previouslydelivered to the user device) to present the advertisement and/or othercontent for display. In one embodiment, the package may include, forexample, HTML, JavaScript, JavaScript Object Notation (“JSON”), andimage assets that can be assembled by the user device according to codein the package (e.g., according to the HTML, JavaScript and JSONportions of the package) for display over an image or other pageelement. The package may include (such as in a JSON portion of thepackage) advertisement tracking data (including, for example, trackingpixels, which are known in the art), as well as information regarding aURI to be requested if the advertisement is “clicked” or otherwiseselected by a user. In some embodiments, the package may includeadditional information, such as video or animation data, sound files,and/or links to additional third-party hosted content for incorporationinto an overlay content display. Overlay content packages are describedfurther in co-pending U.S. patent application Ser. No. 15/951,762, filedApr. 12, 2018 and entitled “MAINTAINING PAGE INTERACTION FUNCTIONALITYWITH OVERLAY CONTENT,” which is hereby incorporated by reference in itsentirety herein.

The advertisement selection system 122 may (6) send to the user device102A the advertisement content package, including the additional codethat, when executed, will cause the user device (e.g., via a browser orother application operating on the user device) to display overlaycontent at a specific position on the page (where the advertisementcontent may be overlaid on an image or video within the page, in someembodiments). As one example, the user device 102A may cause display ofan advertisement or other overlay content over a portion of an image,video, page portion (such as may be identified by a <div> tag in HTML ofthe page) or other element of the page. Displaying the advertisement mayinclude executing additional code in a received data package (incombination with execution of code previously resident on the pageand/or previously delivered by a server, in some embodiments). In thecase of an image as the underlying page element, executing the code maycause the client device to place a container or drawing canvas over theimage or a portion of the image within the page. The code may then causethe user device to render the HTML content of the overlay contentpackage (potentially including executing other embedded or associatedcode) within the newly created container.

The client device may present image or video data from the overlaycontent package for display within the container, such that an overlayimage or video appears to be on top of the underlying page image orwithin the underlying page image. While examples of overlay content thatappears as a banner or rectangle over a bottom portion of an image arediscussed below with respect to FIG. 4B, many other forms of overlaycontent may be displayed in certain embodiments. More details regardingdisplaying overlay content and handling user interaction with overlaycontent are described in co-pending U.S. patent application Ser. No.15/951,762, incorporated by reference above.

While FIG. 1B has been described with reference to an embodiment inwhich the contextual intelligence system 130 generates contextualinformation in response to advertisement requests from user devices, thecontextual intelligence system may additionally or alternativelygenerate contextual information regarding pages in other manners in someembodiments. As one example, in some instances, page URIs may be passedbetween advertising companies and/or other entities in programmatic bidrequests, and one of these entities may wish to utilize the contextualintelligence system to obtain contextual information regarding theURI(s) (such as to use the contextual information in association withthe entity's bid determinations). In such an example, one or more ofthese companies or entities may send a URI to the contextualintelligence system (such as via an API call) as part of a request toreceive contextual data in return, which may cause the contextualintelligence system to generate contextual information for the URI asdescribed above. As another alternative, a publisher's contentmanagement system or other computing system or server may interface withthe contextual intelligence system to associate contextual data with thepublishers' pages as they are published. In such an example, thecontextual intelligence system may generate contextual informationregarding a page prior to any user requesting an advertisement for thatpage (with this contextual information then being retrieved for use oncethe first viewing user's request for an advertisement is received).

FIG. 1C provides an illustrative data flow for performing contextualpage analysis using a contextual intelligence system as describedherein, according to some embodiments. The illustrative flow begins with(1) the contextual analysis API 136 of the contextual intelligencesystem 130 receiving a request for contextual information for a pageidentified by a URI included within the request. The request may be, forexample, an API call made by the advertisement selection system 122. Thecontextual analysis API component 136 of the contextual intelligencesystem may generally be responsible for receiving and responding to APIrequests, as well as initiating and merging results of the various othercomponents 132, 134 and 138 of the contextual intelligence system togenerate the responsive data. FIG. 1C will be described with respect toan instance in which the contextual intelligence system 130 completes afull contextual analysis of a given page. In other instances, as will bedescribed below with respect to FIG. 2, a previously performedcontextual analysis of a page may be reused for ad selection associatedwith a subsequent page view.

The contextual analysis API (2) initiates the analysis by causing thecontextual intelligence system to initiate execution of the headlesscontent extractor 132 to extract content of the page for furtheranalysis. For example, the headless content extractor may request theHTML or other initial file at the provided URI, as hosted at an externalserver. The headless content extractor may then generate final HTML codeor other page representation based on a final state of a headlessbrowser after loading additional content referenced within the page andexecuting associated code (such as JavaScript) included in the initialpage content. As discussed, the headless browser operated in associationwith the headless content extractor may include existing “ad blocker”features so that content such as advertisements that may changefrequently depending on the user viewing the page is not loaded. As isknown in the art, ad blocker functionality may operate by storingvarious rules that block content by its source (such as blockingoutgoing HTTPS requests associated with known ad platforms) or byspecific identified code (such as identifying known scripts that variousadvertisement platforms instruct publishers to include in their pages),among other methods.

Implementing aspects of a headless browser in association with theheadless content extractor may provide various advantages over simplypassing the initial page information of the initial returned HTML fileat the provided URI to the components 134 and 138. For example, variousscripts or code (such as JavaScript libraries used to reference andimplement various third-party functionality integrated within the page)may be embedded or referenced within the initial HTML content, whichupon execution by a browser (whether a standard browser or headlessbrowser) may cause additional text, multimedia and/or other content tobe loaded as part of the page. By considering this additional text,additional images, and/or other additional information that would bedisplayed to a user in a real world viewing instance, the contextualintelligence system may generate more accurate and complete contextualinformation that represents context of the page as it would typically bedisplayed as opposed to context as the page exists in its initial HTMLstate.

Once a final representation of the page content has been generated bythe headless content extractor 132, the extracted HTML and/or other textmay be provided to the natural language processing component 134 at step(4A). In parallel or sequentially, the extracted images and/or othermultimedia content (such as images, video, and/or other visual contentthat would be displayed to a user as part of the loaded page) may beprovided to the computer vision system 138 at step (4B). The naturallanguage processing component may generate (and provide to the API 136)various inferences or other results determined based on the extractedHTML and/or other text. In parallel or sequentially, the computer visionsystem 138 may generate (and provide to the API 136) various computervision inferences or other results determined based on the extractedimages, video and/or other media. The content passed to the naturallanguage processing component 134 and/or other text analysis componentmay include text content other than text and HTML within the pageitself, such as including the page title, headline, and/or variousmetadata. The images and/or videos passed to the computer vision systemmay be only a subset of the total images and/or videos within the page,such as only those meeting a minimum size threshold or other criteria.

The natural language processing component 134 may employ various naturallanguage processing techniques, machine learning techniques, and/orother approaches to determining context of a page's text and outputdeterminations of information such as safety threats (such as whetherthe page is inappropriate for ads, includes adult content, illegalcontent, etc.), content categories or keywords (such as thoserepresenting topics or subject matter discussed in the text content),event categories or keywords (such as real world events discussed in thetext, such as an upcoming sports event or political event), and/or otherkeywords associated with the text content. In some embodiments, theevent categories may be determined using one or more methods describedin co-owned U.S. application Ser. No. 15/602,706, entitled “AutomatedClassification of Network-Accessible Content Based on Events,” filed May23, 2017, which is hereby incorporated by reference herein. The naturallanguage processing component 134 may additionally analyze sentiment ofthe content in addition to subject matter.

The methods employed by the computer vision system 138 to analyze theextracted images or other media may include various machine learningtechniques to identify objects depicted in the media, such asconvolutional neural networks trained to identify company logos and realworld objects depicted in the media. Objects detected (such as a certaincompany's logo, a real world object such as a particular model car, acertain celebrity, a real world location, an animal, faces, and/or manyothers) may be output by the computer vision system 138 as a certainclassification label associated with a corresponding keyword orcategory. In some embodiments, the images may be analyzed to determinebrand safety, which may be based in part on a level of nudity or skinshowing in the images. Some such techniques for determining image safetyare described in co-owned U.S. application Ser. No. 14/196,695, entitled“Systems and Methods for Determining Image Safety,” filed Mar. 4, 2014,which is hereby incorporated by reference herein.

The results from the computer vision system 138 and the natural languageprocessing component 134 may be merged and/or otherwise considered incombination to produce a responsive object at step (6) that is returnedas responsive contextual information at step (7). For example, thecontextual information may include lists of keywords and/or categories,and may additionally include Boolean values (such as whether the contentis safe for advertisements), enumerated values (such as a brand safetyvalue such as “very safe,” “moderately safe,” “unsafe”), and/or numericvalues (such as a safety score or contextual relevancy score). Theresults may be returned to the advertisement selection system 122 and/orstored in association with the URI of the analyzed page in data store145 and/or data store 124.

FIG. 2 is a flow diagram of an illustrative method 200 for determiningwhether to perform contextual page analysis for a specific page prior toselecting an advertisement, according to some embodiments. The method200 may be performed, in some embodiments, by the advertisementcomputing system 120. Individual blocks of method 200 may be performedby the advertisement system 122 or the contextual intelligence system130, depending on the embodiments.

The method 200 begins at block 202, where the advertisement computingsystem 120 receives a request for an advertisement or other supplementalcontent to be displayed within a page having a specific URI identifiedin the request. For example, the request may be a result of a particularuser computing device loading an HTML file (retrieved from the specifiedURI) for display and requesting an advertisement to display inconjunction with the page display. At decision block 204, theadvertisement computing system 120 may determine whether the pagecontent of the page at the specified URI has been recently analyzed bythe contextual intelligence system (e.g., whether there is alreadyinformation representing the results of a contextual analysis of thegiven page stored in a data store for reuse). In some embodiments,contextual page analysis results for a given URI may be stored for a settime, such as one day, one week, one month, or other time period, beforebeing either deleted or refreshed with an analysis of a current versionof the page.

If the advertisement computing system 120 determines at block 204 thatno cached context information is stored for the given URI, the methodmay proceed to block 210 where the advertisement computing system 120adds the URI to a queue for contextual analysis using the headlesscontent extractor 132. Subsequently (which may be immediately in someinstances, or may be after some delay or initiation of a batchprocessing procedure), the advertisement computing system 120 mayperform the contextual analysis of the page and save the results in datastore 145 at block 212. In instances in which the contextual analysiswill not be performed immediately or in substantially real time withrespect to a current viewing of the page by a particular user, the givenuser may not receive any ad content for display.

If instead the advertisement computing system 120 determines at block204 that cached context information is stored for the given URI (and hasnot expired), the method may proceed to block 206, where theadvertisement computing system 120 retrieves the prior contextualanalysis of the page as stored in data store 145. At block 208, theadvertisement computing system 120 may then select an ad or othersupplemental content for display in association with the page based onthe retrieved contextual analysis.

FIG. 3 illustrates a general architecture of a computing environment300, according to some embodiments. As depicted in FIG. 3, the computingenvironment 300 may include a computing system 302. The generalarchitecture of the computing system 302 may include an arrangement ofcomputer hardware and software components used to implement aspects ofthe present disclosure. The computing system 302 may include many more(or fewer) elements than those shown in FIG. 3. It is not necessary,however, that all of these generally conventional elements be shown inorder to provide an enabling disclosure. In some embodiments, thecomputing system 302 may be an example of what is referred to as theadvertisement computing system above.

As illustrated, the computing system 302 includes a processing unit 306,a network interface 308, a computer readable medium drive 310, aninput/output device interface 312, an optional display 326, and anoptional input device 328, all of which may communicate with one anotherby way of a communication bus 336. The processing unit 306 maycommunicate to and from memory 314 and may provide output informationfor the optional display 326 via the input/output device interface 312.The input/output device interface 312 may also accept input from theoptional input device 328, such as a keyboard, mouse, digital pen,microphone, touch screen, gesture recognition system, voice recognitionsystem, or other input device known in the art.

The memory 314 may contain computer program instructions (grouped asmodules or components in some embodiments) that the processing unit 306may execute in order to implement one or more embodiments describedherein. The memory 314 may generally include RAM, ROM and/or otherpersistent, auxiliary or non-transitory computer-readable media. Thememory 314 may store an operating system 318 that provides computerprogram instructions for use by the processing unit 306 in the generaladministration and operation of the computing system 302. The memory 314may further include computer program instructions and other informationfor implementing aspects of the present disclosure. For example, in oneembodiment, the memory 314 may include a user interface module 316 thatgenerates user interfaces (and/or instructions therefor) for displayupon a computing system, e.g., via a navigation interface such as abrowser or application installed on the computing system 302 or theclient computing system 303.

In some embodiments, the memory 314 may include an advertisementselection module 320 and content analysis module 322, which may beexecuted by the processing unit 306 to perform operations according tovarious embodiments described herein. The modules 320 and/or 322 mayaccess the data store 330 in order to retrieve data described aboveand/or store data. The data store may be part of the computing system302, remote from the computing system 302, and/or may be a network-basedservice. The advertisement data store 330 may store variousadvertisement or other overlay content and information, such as images,video, text, associated rules for when to display given overlay content,bid information, keyword information, code packages that instruct abrowser or application how to display given overlay content, and/orother data or content.

In some embodiments, the network interface 308 may provide connectivityto one or more networks or computing systems, and the processing unit306 may receive information and instructions from other computingsystems or services via one or more networks. In the example illustratedin FIG. 3, the network interface 308 may be in communication with aclient computing system 303 via the network 336, such as the Internet.In particular, the computing system 302 may establish a communicationlink 342 with a network 336 (e.g., using known protocols) in order tosend communications to the computing system 303 over the network 336.Similarly, the computing system 303 may send communications to thecomputing system 302 over the network 336 via a wired or wirelesscommunication link 340. In some embodiments, the computing system 302may additionally communicate via the network 336 with an optionalthird-party advertisement service 301, which may be used by thecomputing system 302 to retrieve advertisement data to be delivered to aclient computing system 303. Depending on the embodiment, the computingsystem 302 may be configured to retrieve advertisement data from eitheradvertisement data store 330 or third-party advertisement service 301depending on various information, such as publisher page information,keywords, image information, advertiser preference, comparison of bidinformation, and/or other factors. The computing system 302 mayadditionally be in communication with a publisher computing system 305and/or other third-party servers to retrieve content to be analyzed forcontext as described herein.

Those skilled in the art will recognize that the computing systems 302and 303 may be any of a number of computing systems including, but notlimited to, a laptop, a personal computer, a personal digital assistant(PDA), a hybrid PDA/mobile phone, a mobile phone, a smartphone, awearable computing device, an electronic book reader, a digital mediaplayer, a tablet computer, a gaming console or controller, a kiosk, anaugmented reality device, another wireless device, a set-top or othertelevision box, one or more servers, and the like. The client computingsystem 303 may include similar hardware to that illustrated as beingincluded in computing system 302, such as a display, processing unit,network interface, memory, operating system, etc. In some embodiments,the client computing system 303 may perform a number of featuresdescribed above (such as displaying a page, requesting overlay content,supplementing display of the page with overlay content, etc.) based inpart on a browser or other application operating on the client computingsystem 303 executing code received over the network from the computingsystem 302, the publisher computing system 305, and/or othernetwork-accessible server or service.

FIGS. 4A and 4B illustrate a displayed page 400A prior to selection ofan advertisement (in FIG. 4A) and subsequent to an overlay advertisement410 being displayed over the underlying page image 404B (in FIG. 4B). Inthe illustrated example, the page is displayed in a browser window. Thepage includes lines of text (represented in the figures as lines), aswell as images. As will be appreciated, a page may contain many othertypes of content.

FIG. 4A may represent the page 400A at the time it is first displayed ona client computing system. The page includes a first image 404A(including rendered text within the image represented as lines) as wellas a second image. Each of the two images may be considered a differentelement of page 400A. These images, as well as others on the page thatare not in view, may be identified by code of the page upon initial pageload, such as by code analyzing the page and identifying referenceswithin the HTML to URIs of those page images.

FIG. 4B may represent the same page as in FIG. 4A above, but may befollowing an update to the display of the page (which may be less than asecond after the initial page load, in some embodiments) to incorporatean advertisement 410 that was selected by an advertisement service basedat least in part on the contextual analysis approaches described herein.As illustrated, overlay content 410 in the form of an advertisement hasbeen added over a portion of image 404B. As will be appreciated by oneof ordinary skill in the art, the overlay content 410 may be selectableby a user to navigate to a landing page associated with theadvertisement, and the advertisement may move with the image (such thatit appears to be pegged to the image or part of the image) as the usercontinues to scroll through the page.

While in-image advertisements are described above as examples, it willbe appreciated that aspects of the disclosure provide improvements formany other types of overlay content. For example, overlay content asdescribed herein may be presented over video content, Flash objects, aportion of a page itself, or other content of a webpage or userinterface besides a static image. In one embodiment, overlay contentthat is bound to a portion of a page itself rather than to a singlemedia object may be configured to visually collapse or minimize after apredetermined time (such as five seconds), or in response to a userrequest to hide the overlay content.

It is to be understood that not necessarily all objects or advantagesmay be achieved in accordance with any particular embodiment describedherein. Thus, for example, those skilled in the art will recognize thatcertain embodiments may be configured to operate in a manner thatachieves or optimizes one advantage or group of advantages as taughtherein without necessarily achieving other objects or advantages as maybe taught or suggested herein.

All of the processes described herein may be embodied in, and fullyautomated via, software code modules executed by a computing system thatincludes one or more general purpose computers or processors. The codemodules may be stored in any type of non-transitory computer-readablemedium or other computer storage device. Some or all the methods mayalternatively be embodied in specialized computer hardware. In addition,the components referred to herein may be implemented in hardware,software, firmware or a combination thereof.

Many other variations than those described herein will be apparent fromthis disclosure. For example, depending on the embodiment, certain acts,events, or functions of any of the algorithms described herein can beperformed in a different sequence, can be added, merged, or left outaltogether (e.g., not all described acts or events are necessary for thepractice of the algorithms). Moreover, in certain embodiments, acts orevents can be performed concurrently, e.g., through multi-threadedprocessing, interrupt processing, or multiple processors or processorcores or on other parallel architectures, rather than sequentially. Inaddition, different tasks or processes can be performed by differentmachines and/or computing systems that can function together.

The various illustrative logical blocks, modules, and algorithm elementsdescribed in connection with the embodiments disclosed herein can beimplemented as electronic hardware, computer software, or combinationsof both. To clearly illustrate this interchangeability of hardware andsoftware, various illustrative components, blocks, modules, and elementshave been described above generally in terms of their functionality.Whether such functionality is implemented as hardware or softwaredepends upon the particular application and design constraints imposedon the overall system. The described functionality can be implemented invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the disclosure.

The various illustrative logical blocks and modules described inconnection with the embodiments disclosed herein can be implemented orperformed by a machine, such as a processing unit or processor, adigital signal processor (DSP), an application specific integratedcircuit (ASIC), a field programmable gate array (FPGA) or otherprogrammable logic device, discrete gate or transistor logic, discretehardware components, or any combination thereof designed to perform thefunctions described herein. A processor can be a microprocessor, but inthe alternative, the processor can be a controller, microcontroller, orstate machine, combinations of the same, or the like. A processor caninclude electrical circuitry configured to process computer-executableinstructions. In another embodiment, a processor includes an FPGA orother programmable device that performs logic operations withoutprocessing computer-executable instructions. A processor can also beimplemented as a combination of computing devices, e.g., a combinationof a DSP and a microprocessor, a plurality of microprocessors, one ormore microprocessors in conjunction with a DSP core, or any other suchconfiguration. Although described herein primarily with respect todigital technology, a processor may also include primarily analogcomponents. For example, some or all of the signal processing algorithmsdescribed herein may be implemented in analog circuitry or mixed analogand digital circuitry. A computing environment can include any type ofcomputer system, including, but not limited to, a computer system basedon a microprocessor, a mainframe computer, a digital signal processor, aportable computing device, a device controller, or a computationalengine within an appliance, to name a few.

The elements of a method, process, or algorithm described in connectionwith the embodiments disclosed herein can be embodied directly inhardware, in a software module stored in one or more memory devices andexecuted by one or more processors, or in a combination of the two. Asoftware module can reside in RAM memory, flash memory, ROM memory,EPROM memory, EEPROM memory, registers, hard disk, a removable disk, aCD-ROM, or any other form of non-transitory computer-readable storagemedium, media, or physical computer storage known in the art. An examplestorage medium can be coupled to the processor such that the processorcan read information from, and write information to, the storage medium.In the alternative, the storage medium can be integral to the processor.The storage medium can be volatile or nonvolatile.

Conditional language such as, among others, “can,” “could,” “might” or“may,” unless specifically stated otherwise, are otherwise understoodwithin the context as used in general to convey that certain embodimentsinclude, while other embodiments do not include, certain features,elements and/or steps. Thus, such conditional language is not generallyintended to imply that features, elements and/or steps are in any wayrequired for one or more embodiments or that one or more embodimentsnecessarily include logic for deciding, with or without user input orprompting, whether these features, elements and/or steps are included orare to be performed in any particular embodiment.

Disjunctive language such as the phrase “at least one of X, Y, or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y, or Z, or any combination thereof (e.g., X, Y, and/or Z).Thus, such disjunctive language is not generally intended to, and shouldnot, imply that certain embodiments require at least one of X, at leastone of Y, or at least one of Z to each be present.

Any process descriptions, elements or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or elements in the process. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, executedout of order from that shown, or discussed, including substantiallyconcurrently or in reverse order, depending on the functionalityinvolved as would be understood by those skilled in the art.

Unless otherwise explicitly stated, articles such as “a” or “an” shouldgenerally be interpreted to include one or more described items.Accordingly, phrases such as “a device configured to” are intended toinclude one or more recited devices. Such one or more recited devicescan also be collectively configured to carry out the stated recitations.For example, “a processor configured to carry out recitations A, B andC” can include a first processor configured to carry out recitation Aworking in conjunction with a second processor configured to carry outrecitations B and C.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure.

What is claimed is:
 1. A computer system comprising: memory; and aprocessor in communication with the memory and configured withprocessor-executable instructions to perform operations comprising:receiving, at a server, an application programming interface (API)request to perform contextual analysis of a page, wherein the page isidentified by a uniform resource identifier (URI); initiating executionof a headless content extractor by the server, wherein the headlesscontent extractor is configured to extract content included in orreferenced in the page, wherein the content includes text formatted in aHyperText Markup Language (HTML) format and one or more images, whereinat least a portion of the content extracted is not included in aninitial file stored at the URI but is requested from an external systemover a network based on at least one of (a) execution by the headlesscontent extractor of code included in the initial file or (b) areference in HTML of the initial file to an additional URI; performingcontextual analysis of the content extracted by the headless contentextractor, wherein performing the contextual analysis comprises (c)analyzing text of the content using natural language processing and (d)analyzing the one or more images using one or more computer visiontechniques; and generating a response to the API request that includesresults of the contextual analysis, wherein the results are based atleast in part on the analyzing of the text and the analyzing of the oneor more images.
 2. The computer system of claim 1, wherein the contentextracted by the headless content extractor includes one or more videos,and wherein performing the contextual analysis of the content includesanalyzing at least a portion of the one or more videos using one or moremachine learning models that implement computer vision techniques. 3.The computer system of claim 1, wherein the results of the contextualanalysis included in the response to the API request comprise one ormore keywords.
 4. The computer system of claim 3, wherein the one ormore keywords include at least one of (a) a topic discussed in the textof the content extracted by the headless content extractor, (b) a realworld event referenced in the text of the content extracted by theheadless content extractor, or (c) a name of an object depicted in theone or more images.
 5. The computer system of claim 1, wherein theheadless content extractor comprises a headless browser that processesat least HTML of the page to produce output without causing display ofthe output.
 6. The computer system of claim 5, wherein the headlessbrowser blocks advertisement content, such that the content extracted bythe headless content extractor excludes content hosted by or associatedwith an advertisement service.
 7. The computer system of claim 1,wherein the processor is further configured to perform additionaloperations comprising: receiving an indication that the page is beingviewed at a user device; selecting an advertisement to present on thepage based at least in part on the results of the contextual analysis;assembling a package that includes one or more media assets andadditional code configured for execution at the user device and whichcauses the user device to present the advertisement for display withinthe page; and sending, over a network, the package to the user device.8. The computer system of claim 7, wherein the package comprises HTML,JavaScript, JavaScript Object Notation (“JSON”) data, and one or moreimages that are positioned for display by the user device according tothe additional code in the package.
 9. The computer system of claim 1,wherein the processor is further configured to perform additionaloperations comprising: determining that the page is not appropriate forpresenting advertisements based at least in part on contextual analysisof the portion of the content that is not included in the initial filestored at the URI but is requested from the external system, wherein thepage is determined not to be appropriate for presenting advertisementsbased on a determination that the portion of the content includes atleast one of adult content or illegal content; and including, in theresponse to the API request, a value indicating that the page is notappropriate for presenting advertisements.
 10. A computer-implementedmethod comprising: receiving, at a server, a request to performcontextual analysis of a page, wherein the page is identified by auniform resource identifier (URI); initiating execution of a headlesscontent extractor by the server, wherein the headless content extractoris configured to extract content included in or referenced in the page,wherein the content includes text formatted in a HyperText MarkupLanguage (HTML) format and one or more images, wherein at least aportion of the content extracted is not included in an initial filestored at the URI but is requested from an external system over anetwork based on at least one of (a) execution by the headless contentextractor of code included in the initial file or (b) a reference inHTML of the initial file to an additional URI hosted by the externalsystem; performing contextual analysis of the content extracted by theheadless content extractor, wherein performing the contextual analysiscomprises (c) analyzing text of the content using natural languageprocessing and (d) analyzing the one or more images using one or morecomputer vision techniques; and storing, in a data store accessible tothe server, keywords associated with the page determined from theresults of the contextual analysis, wherein the keywords are based atleast in part on the analyzing of the text and the analyzing of the oneor more images, wherein the keywords are associated in the data storewith the URI.
 11. The computer-implemented method of claim 10, whereinthe content extracted by the headless content extractor includes one ormore videos, and wherein performing the contextual analysis of thecontent includes analyzing at least a portion of the one or more videosusing one or more machine learning models that implement computer visiontechniques.
 12. The computer-implemented method of claim 10, wherein theone or more keywords include at least one of (a) a topic discussed inthe text of the content extracted by the headless content extractor, (b)a real world event referenced in the text of the content extracted bythe headless content extractor, or (c) a name of an object depicted inthe one or more images.
 13. The computer-implemented method of claim 10,wherein the headless content extractor comprises a headless browser thatprocesses at least HTML of the page to produce output without causingdisplay of the output.
 14. The computer-implemented method of claim 13,wherein the headless browser blocks advertisement content, such that thecontent extracted by the headless content extractor excludes contenthosted by or associated with an advertisement service.
 15. Thecomputer-implemented method of claim 10 further comprising: receiving anindication that the page is being viewed at a user device; selecting anadvertisement to present on the page based at least in part on theresults of the contextual analysis; assembling a package that includesone or more media assets and additional code configured for execution atthe user device and which causes the user device to present theadvertisement for display within the page; and sending, over a network,the package to the user device.
 16. The computer-implemented method ofclaim 10 further comprising: determining that the page is notappropriate for presenting advertisements based at least in part oncontextual analysis of the portion of the content that is not includedin the initial file stored at the URI but is requested from the externalsystem; and storing, in the data store accessible to the server, a valueindicating that the page is not appropriate for presentingadvertisements.
 17. The computer-implemented method of claim 16 furthercomprising: receiving, at the server, a subsequent request for anadvertisement to present at a client device in association with thepage, wherein the subsequent request identifies the page by the URI;retrieving from the data store, based on the URI, the value indicatingthat the page is not appropriate for presenting advertisements; anddetermining not to send any advertisement content to the client devicein response to the subsequent request based on the value.
 18. Acomputer-readable, non-transitory storage medium storing computerexecutable instructions that, when executed by one or more computersystems, configure the one or more computer systems to performoperations comprising: receiving a request to perform contextualanalysis of a page, wherein the page is accessible over a network via auniform resource identifier (URI); initiating execution of a headlesscontent extractor, wherein the headless content extractor is configuredto extract content included in or referenced in the page without causingdisplay of the content, wherein the content includes text formatted in aHyperText Markup Language (HTML) format and one or more images, whereinat least a portion of the content extracted is not included in aninitial file stored at the URI but is requested from an external systemover the network based on each of (a) execution by the headless contentextractor of code included in the initial file and (b) a reference inHTML of the initial file to an additional URI hosted by the externalsystem; performing contextual analysis of the content extracted by theheadless content extractor, wherein performing the contextual analysiscomprises (c) analyzing text of the content using natural languageprocessing and (d) analyzing the one or more images using one or morecomputer vision techniques; and storing, in an electronic data store,keywords associated with the page determined from the results of thecontextual analysis, wherein the keywords are based at least in part onthe analyzing of the text and the analyzing of the one or more images,wherein storing the keywords includes associating the keywords with theURI in the electronic data store.
 19. The computer-readable,non-transitory storage medium of claim 18, wherein the computerexecutable instructions implement at least a portion of an applicationprogramming interface (API) configured to receive requests from one ormore advertising platforms, wherein the keywords are provided via theAPI to a first advertising platform in response to a programmatic bidrequest for an advertisement received from the first advertisingplatform.
 20. The computer-readable, non-transitory storage medium ofclaim 18, wherein the analyzing the one or more images comprisesprocessing features of the image data of the one or more images withinone or more convolutional neural networks trained to identify objects inimages, wherein output of the convolutional neural networks comprises aclassification label associated with one of the keywords associated withthe page.